Sparse Coding Models Can Exhibit Decreasing Sparseness while Learning Sparse Codes for Natural Images
نویسندگان
چکیده
The sparse coding hypothesis has enjoyed much success in predicting response properties of simple cells in primary visual cortex (V1) based solely on the statistics of natural scenes. In typical sparse coding models, model neuron activities and receptive fields are optimized to accurately represent input stimuli using the least amount of neural activity. As these networks develop to represent a given class of stimulus, the receptive fields are refined so that they capture the most important stimulus features. Intuitively, this is expected to result in sparser network activity over time. Recent experiments, however, show that stimulus-evoked activity in ferret V1 becomes less sparse during development, presenting an apparent challenge to the sparse coding hypothesis. Here we demonstrate that some sparse coding models, such as those employing homeostatic mechanisms on neural firing rates, can exhibit decreasing sparseness during learning, while still achieving good agreement with mature V1 receptive field shapes and a reasonably sparse mature network state. We conclude that observed developmental trends do not rule out sparseness as a principle of neural coding per se: a mature network can perform sparse coding even if sparseness decreases somewhat during development. To make comparisons between model and physiological receptive fields, we introduce a new nonparametric method for comparing receptive field shapes using image registration techniques.
منابع مشابه
Face Recognition using an Affine Sparse Coding approach
Sparse coding is an unsupervised method which learns a set of over-complete bases to represent data such as image and video. Sparse coding has increasing attraction for image classification applications in recent years. But in the cases where we have some similar images from different classes, such as face recognition applications, different images may be classified into the same class, and hen...
متن کامل1 Sparse Codes for Natural Images
The human visual system, at the primary cortex, has receptive fields that are spatially localized, oriented and bandpass. It has been shown that a certain learning algorithm to produce sparse codes for natural images leads to basis functions with similar properties. This learning algorithm optimizes a cost function that trades off representation quality for sparseness, and searches for sets of ...
متن کاملSparse Codes for Natural Images
The human visual system, at the primary cortex, has receptive fields that are spatially localized, oriented and bandpass. It has been shown that a certain learning algorithm to produce sparse codes for natural images leads to basis functions with similar properties. This learning algorithm optimizes a cost function that trades off representation quality for sparseness, and searches for sets of ...
متن کاملNatural Image Statistics and Eecient Coding
Natural images contain characteristic statistical regularities that set them apart from purely random images. Understanding what these regularities are can enable natural images to be coded more eeciently. In this paper, we describe some of the forms of structure that are contained in natural images, and we show how these are related to the response properties of neurons at early stages of the ...
متن کاملSparse coding in striate and extrastriate visual cortex.
Theoretical studies of mammalian cortex argue that efficient neural codes should be sparse. However, theoretical and experimental studies have used different definitions of the term "sparse" leading to three assumptions about the nature of sparse codes. First, codes that have high lifetime sparseness require few action potentials. Second, lifetime-sparse codes are also population-sparse. Third,...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره 9 شماره
صفحات -
تاریخ انتشار 2013